ISIS: A Multi-Modal, Trilingual, Distri developed with CORBA, Java

ISIS (Intelligent Speech for Information Systems) is a trilingual spoken dialog system in the stocks domain. It supports the three languages commonly used in Hong Kong (Cantonese, Putonghua and English), and serves as a test-bed for our research in various speech and language technologies. ISIS also features combined interaction and delegation dialogs, and automatic assimilation of newly listed stock names into the system’ s knowledge base. This paper focuses on the architecture and multi-modality of ISIS. We use the CORBA middleware to implement a distributed system that is interoperable across platforms. We also describe the incorporation of KQML (Knowledge Query and Manipulation Language) software agents in ISIS to handle delegation dialogs. The latest enhancement supports multi-modal and mixed-modal input which suit the natural affordances of certain interactions in order to improve usability. Input modalities include speaking, typing or mouse-clicking. Output media include synthesized speech, text, tables and graphics.